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ABSTRACT 

The Students of Limited English Proficiency (SLSP) 
program of the Honolulu school District (Hawaii) is designed to help 
the large number of limited-English-speaking immigrants adji,;st to the 
American culture in the Hawaiian setting by acquiring basic 
communication skills for regular classroom participation and school 
activities. The program is implemented in 54 schools with over 60 
permanent teachers, 10 educational assistants, and over 80 part-time 
teachers. Students are selected for participation and exit based on 
language dominance and proficiency ratings. Program components 
include: (l) a whole language approach emphasizing communication 
skills; (2) a comprehensive database containing achievement status 
and diagnostic data, norm-referenced, criterion-referenced, and 
non-test; (3) a program implementation checklist and quality control 
system; (4) an extensive staff development program; and (5) a 
survival skills segment for newly-arrived immigrants, specific 
learner objectives include: making significant gains in oral 
language, reading, language arts, and mathematics; adjusting to 
school and community, and exiting the program at the 25th percentile 
or higher on a standardized achievement test. Evaluation of the 
program began only in 1985, with an extensive effort to develop a 
longitudinal database for student tracking, to initiate a quality 
monitoring process, and to implement systematic program evaluation. 
(MSE) 
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The SLEP Program 

Located In the "melting pot of the Pacific," the Hawaii school system serves a population of varied 
ethnic, cultural and linguistic backgrounds. A large percentage of its 200,000 students are either 
recent Immigrants or second-generation children of non-English speaking parents. The Honolulu 
School District Is the largest of seven school districts In Hawaii. It has 55 public schools with an 
enrollment of over 35,000, including a sizeable number of students whose dominant language Is 
not English. Their limitation In the use of English prevents these students from functioning 
effectively In the regular classroom. The Students of Limited English Proficiency (SLEP) Program 
is designed to overcome this language barrier. The overall objective of the program Is to help 
these students to adjust to the American culture in the Hawaiian setting. The program enables 
them to acquire basic communication skills so that they can participate in the regular classroom 
instruction and school activities appropriate for their age and grade level. 

The program Is implemented in 54 schools in the district with an annual enrollment of 
approximately 4,000 In grades K - 12. Program staff Include over 60 permanent teachers, 10 
educational assistants, and over 80 part-time temporary teachers. Administrative and Instructional 
leadership is provided by a district educational specialist and two resource teachers at the district 
level. Program funding Is approximately $2.2 million a year. 

Students are selected for program participation on the basis of language dominance ratings and 
performance on standardized norm-referenced tests. Generally, low language dominance ratings 
and low performance (i.e., below the 25th percentile) on norm-referenced tests qualify a student for 
program participation. Students exit the program when they achieve a high language dominance 
rating and score at the 25th percentile or above on norm-referenced tests in reading, language and 
mathematics, 

The program consists of a well-integrated set of essential components: 

0 The whole language approach Is adopted to emphasize language development and a 
communication skills rather than the mechanics of reading. 

0 A comprehensive database system is established to provide Information on achievement 
status and diagnostic data to the teaching staff. The systen contains both norm- 
referenced and criterion-referenced test and non-test data. 

0 A high fidelity program implementation Is assured through the use of a program 

Implementation checklist (which is administered to program staff at least twice a year) and 
quality monitoring system for Identifying potential problems and developing solutions. 

1 
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0 An extensive staff development program component provides Inservlce training to 
program staff In the relevant content areas (e.g., bilingual and bicultura! education) and 
pedagogical issues (e.g., teaching English as a second language). Parents of program 
sturlents also attend the inservlce training when appropriate. 

0 A survival skills component provides training In cultural and community survival skills to 
enable program students who are newly-arrived Immigrants to effectively adjust to their 
new environment. 

Specific learner objectives include the following: 

1 . Students will make significant gains in oral language as measured by the Basic Inventory 
of Natural Language (BINL) 

2. Students will make significant gains in reading, language arts and mathematics as 
neasured by the Metropolitan Achievement Tests (MAT). 

3. Students will be able to adjust to the school and community as Indicated by the Student 
Record Profile (SRP). 

4. Students will be able to exit the program by scoring at the 25th percentile or higher on the 
MAT reading, language arts and mathematics subtests. 

Although the SLEP Program has been in existence for almost two decades, efforts to assess 
program impact had been sporadic until more recent years. Specifically, in 1985 the district 
embarked on an extensive effort to (a) develop a longitudinal database to track program 
Implementation and Its impact on student performance, (b) Initiate a quality monitoring process to 
provide data-based technical assistance for program improvement, and (c) implement a 
systematic program evaluation process to document student performance and provide 
suggestions for program improvement. These critically important program processes are 
described below. 



The Longitudinal Database 

The development of a program database was guided by considerations relating to program 
Implementation, student performance, as well as school, family and community factors. 

Program Implementation 

Program impact Is often assessed without first ascertaining whether a program has been put in 
place (Hall & Loucks, 1977). Such assessments are potentially useless to decisionmakers. Data 
on program Implementation an particularly important in bilingua! education projects because 
these projects face unique difficulties In program Implementation (BIssell, 1979). Information on 
degree of implementation allows us to n-^ake a more valid interpretation of project outcomes. 

In addition, descriptive data on student characteristics, types of services provided, length of 
student participation and criteria for determining language proficiency are useful in project 
management. Program irhpiementation information can also help Identify key factors which 
influence the success of the project. 

Student Performance 

The ultimate beneficiaries of any bilingual program are the participating students. Student 
performance is a critical element of an assessment of program impact. An assessment of student 
performance should include affective and attitudinal changes as well as cognitive skills. 

School, Family and Community Factors 

Other essential elements of a bilingual program pertain to the influence of school, family and 
community factors on program outcomes. School staff, parents and community members vary in 
their attitudes toward particular languages. In their support of bilingual education, and In their 
willingness to promote the use of languages other than English In the classroom (BIssell, 1979). It 
is important that an assessment of program impact takes Into account the roles of parents, the 
community at large and the institutional context in which the program operates. 
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Data Sources 



Program Implementation 

To obtain data on piogram Implementation, a program Implementation checklist was created. Tliis 
instrument assesses the degree to whicli essential program elements are implemented in each 
participating school. The checklist Is administered by disirlct staff who have an Intimate 
knowledge of program operations. The staff are encouraged to use multiple methods (e.g., 
Interview, review of documents, onsite observations) to gather evidence of high (or low) fidelity 
program Implementation. The instrument consists of 24 Items relating to the following aspects of 
program operations: 

0 Instructional approach 

o Diagnosis 

0 Student grouping 

o Monitoring 

o Staff development 

o Parent Involvement 

Each Item is rated a on three-point scale: not implemented, partially implemented or fully 
implemented. 

Student Performance 

The assessment of student achievement includes both norm- and criterion-referenced measures. 
In the norm- referenced mode, a pretest-posttest design Is used with test norms serving as 
controls. In the criterion-referenced mode, pretest-posttest gains are measured without reference 
to a norm group. The primary Instruments consist of the Metropolitan Achievement Tests (1978 
and 1985) and the Basic Inventory of Natural Language (1979). 

In the norm-referenced mode, testing occurs close to the norming dates (October 15 and April 25 
for pretest and posttest, respectively). Whenever possible, posttesting Includes students who have 
exited the program. In the criterion-referenced mode, the pretest is generally adminlsterfod when 
students enter the program. Posttesting generally occurs when students exit the program. 

Achievement status of students with low language dominance ratings, as determined by the state's 
English Proficiency Test, is assessed by the BINL only. Most of these students are immigrants who 
have just arrived from their native countries and are non-readers of English, Students with high 
language dominance ratings are tested with the MAT In a fall-spring testing cycle. 
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In addition to these standardized tests, the program uses a Student Record Profile (SRP) system to 
gather data on specific language objectives completed by each participating student. These 
objectives Include four domains of language skills: syntax, semantic, morphemic and phonic. The 
SRP system also includes data on social adjustment to the school and the community. Four major 
components are: culture, basic survival, school survival and community survival. 

A third measure of student performance consists of school grades of participating students In four 
subject areas: mathematics, language arts, reading and speaking/listening. Since grading 
practices differ from school to school, all grade designations are converted to fit a five-point scale: 

1 = Fallino 

2 = Barely Passing 

3 » Satisfactory 

4 = Very Good 

5 = Excellent 

School, Family and Community Factors 

School or classroom climate has long been identified as a major contributing factor to high student 
achievement, particularly in compensatory education ^Edmonds, 1982; MacKenzle, i983). To 
assess class climate, a classroom climate Inventory was adapted from existing inventories on 
school climate. The Instrument consists of 28 Items relating to both positive and negative school 
affects, including coheslveness, goal-direction, clarity, satisfaction, openness, reward, cooperation, 
communication, warmth, formality, favoritism, apathy and cliqueness. The Items are presented on 
a three-point scale of Yes, Maybe or No. Sample Items include: 

0 I have friends in my class. 

0 There are many rules to follow In my class. 

0 in my class all students do the same things. 

All data collection activities are conducted by the distric; -* .school level staff. For example, school 
level program staff administer both MAT and BINL to the particifjating students first In 
Septpmber/October and then In April/May. The MAT is adminlr*«rod In small groups, the BINL (a 
language production test) is given individually. The class climate Inventory is administered by 
school le/el staff towards the end of the school year. District level staff (resource teachers) make 
onslte visits twice during the school year to a sample of SLEP schools to gather data on program 
implementation. Other evaluation information such as grade point averages and performance on 
specific instructional objectives is gathered from school and program files. 



Building tlie Database 

The evaluation information is first transcribed onto a set of data summary forms at the student, 
classroom and school levels. The data are then entered into the computer for easy storage, 
retrieval and analysis. All data transcription and data entry activities are accomplished at the 
district office. 

The computerized database consists of data files at three different levels: schools, classrooms and 
individual students. School level data files cover the following variables: 

0 School name 

o Type of school (e.g., elementary, secondary) 

0 School enrollment 

0 SLEP enrollment 

0 Language groups 

o School locale (e.g., urban, suburban, rural) 

o Program setting ( e.g., in-class, pull-out) 

0 Program funding 

o Teacher-student ratio 

G Staffing patterns (e.g., ESL and bilingual teachers) 

0 Program implementation status 

0 School climate 
Classroom level data include the following variables: 

0 Grade level 

0 Class size 

0 Instructional setting (e.g., pull-out, in-class) 

0 Teacher-student ratio 

0 Length of instruction (minutec per day or week) 

o Language groups in class 

0 Instructional staff 
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o Instructional service 
0 Use of English In class 
0 TIme-on-task 

o Program Implementation status 
0 Class climate 
Student levil data cover the following variables: 
o Grade level 
o Primary home language 
o Ethnicity 

o Language dominance rating 
o Gender 
o Date of birth 

o Participation In other special programs 
o Date of program exit 
o Achievement on 

- MAT reading 

- MAT language arts 

- MAT mathematics 
-BINL 

o School grades In 

- Mathematics 

- Language arts 

- Reading 

- Speal<ing/llstenlng 

o Student Record Profile 



- Syntax 

• Semantic 

- Morphemic 

- Phonic 

- Culture survival 

- Basic survival 

• School survival 

- Community survival 

These data files provide data elements needed In annual program evaluation and In an on-going 
program quality monitoring process. 
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Systematic Evaluation 



Over the past several years, the program has been the focus of a systematic, comprehensive 
evaluation. The evaluation Is to provide evidence of program impact as well as Information useful 
for program Improvement. While the longitudinal data base provides a wide range of cognitive and 
affective measures, district and school level administrators are particularly concerned about the 
performance of program participants on norm-referenced standardized tests. It Is particularly 
Important that there be evidence that the program is helping to narrow the achievement gap 
between students of limited English proficiency and their monolingual English speaking peers. An 
Implicit assumption is that once these students overcome the language barrier, their performance 
In the regular school program will pose no specific problems. 

The evaluation is conducted with the assistance of an external consultant and follows a set of 
guidelines developed on the basis of relevant research findings (e.g., Bissell, 1979; Berke, 1980; 
Berke, 1983) and experiences in similar compensatory education programs (e.g.. Chapter 1). The 
guidelines help ensure the technical adequacy of the evaluation and its potential usefulness to 
decisionmakers. 

In assessing student achievement on norm-referenced tests, a pretest-posttest design is used with 
test norms serving as controls (Tallmadge, 1982). The Metropolitan Achievement Test is adopted 
for use In the evaluation because It meets the criteria of validity and reliability and has a reasonably 
good match with the program's instructional objectives. The test is administered close to the 
empirical notm dates (October 15 and April 25 tor pretest and posttest, respectively). Students at 
different grades take the respective test levels recommended by the test publisher. Testing 
Includes both current students and those who have exited the program for a year. The latter group 
is included to provide data on the longer-term impact of the program. 

Achievement gains are measured in the normal curve equivalent (NCE) metric. The NCE metric, a 
transformed measure of percentile ranks, assumes that with "regular' schooling, students will 
maintain their relative status, making zero NCE gain. NCE gains can be put in three categories 
(positive, zero and negative) to find out the proportion of program students who make positive 
gains (i.e., making greater progress than their regular peers and narrowing the achievement gap). 

The evaluation Is conducted as a team effort between the external consultant and the district staff 
to promote evaluation use (Dickey. 1980; Eash, 1985). The external consultant provides technical 
guidance and assistance to ensure the integrity and objectivity of the evaluation. The district staff 
perform most of the data collection and preliminary data aggregation. The external consultant Is 
responsible for formal data analysis and the preparation of the evaluation report. 

The evaluation maintains a client orientation and shifts the emphasis from summatlve evaluation to 
a systems approach to program Improvement. It constitutes a continuous activity of data 
collection and analysis for program monitoring and tailoring (Cooley, 1983). 
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Using the longitudinal database, the annual evaluation addresses a wide range of questions, 
including; 

0 What staffing configuration produce the highest students achievement in the SLEP 
program? 

0 Are the instructional strategies being Implemented as Intended? 

0 What instructional strategies and materials are most effective in developing cognitive 
skills? 

o What kinds of inservice training seem most appropriate for the SLEP program staff? 

0 What kinds of Inservice training seem most satisfactory to SLEP parents? 

o Are students exited or mainstreamed from the program performing satisfactorily in the 
regular classroom? 

0 What is the performance pattern of program students over time? 

0 Is the prevailing school climate positive as perceived by program students? 

0 How are program students adjusting to the school and community? 

0 What does a typical project school look like in terms of such characteristics as: 

- Subject areas covered 

- Length of Instruction 

- Language use in class 

- Instructional approach 

- Staff training 

- Staff characteristics 

- Teacher-student ratio 

- Achievement profile 

- Length of student participation 

An annua! evaluation has been conducted on the program since 1985 to address these questions. 
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Quality Monitoring 

Most program impros'ement approaches establish new cooperative diagnostic and problem- 
solving groups to make thii.gs happen. These groups are variously called quality circles, quality of 
(school) life committees, problem-solving task forces or leadership teams (Pratzner, 1984). They 
provide the opportunity for people to kJentIfy barriers to the effectiveness of their organization and, 
through problem-solving, break down those barriers. The approach Is based on the conviction 
that .vlthin the organization there exists a largely untapped pool of creative expertise and insight 
which the existing organizational structure has hidden or suppressed. This latent talent can be 
channeled and put to work to Increase school and program effectiveness. In the Honolulu district, 
such a process Is called quality monitoring. A participatory and collaborative effort, quality 
monitoring seeks to bring about positive changes In the SLEP program, It Is a systematic process 
of examining program Implementation variables and evaluation results and translating them Into 
action plans to Improve student achievement. A dynamic process, quality monitoring promotes 
school Itjvel use of evaluation data for program Improvement, leads to an on-going Wontiflcatlon 
and analysis of variables affecting student achievement and en'iures fidelity of program 
implementation. An Important part of quality monitoring Is the continuous exploration for better 
ways of providing instruction to program participants. The kJentlflcation and analysis of Important 
variables provide a sound basis for examining program Implementation and impact. 

As change facilitators (Mall & Mord, 1984), the district level staff initiate the quality monitoring 
process by assuming the following responsibilities: 

0 Examination of achievement gains for each program school In comparison to previous 
years, to district averages, and to grade level averages. Significant patterns are noted. 

0 Examination and analysis of program implementation variables. 

o Preparation of school level staff and administrators for the initial quality monitoring 
meeting by providing a list of topics to be discussed. 

o Coordination of meetings with school level staff and administrators to identify variables 
and practices in the areas of administration, program implementation, monitoring, testing 
and evaluation, curriculum and Inservice, and parent Involvement. 

As program implementation and outcome variables are reviewed, questions such as the following 
are raised: 

0 Does the staffing configuration produce a classroom environment conducive to maximal 
language learning? 

0 Is the instructional delivery system meeting the needs of program participants? 

0 Is the per pupl! cost noticeably higher or lower than others? Why? 

o Are the evaluation procedures appropriate for the target population? 

0 Are the evaluation results significantly better or worse than those of previous years? 

0 Do the evaluation results indicate significant patterns his year, from year to year, and/or 
over the years? 

0 What are the strong and weak areas in the test performance of program participants? 

11 
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0 Are program staff using appropriate Instructional strategies and materials? 
o What are the Inservlce needs of r^rogram staff? 

0 Are parent Involvement activities helping parents to help their children at home? 

Then, In a collaborative, problem solving mode, district and school level staff and administrators 
develop a plan of action and Identify persons responsible for carrying the plan. Decision making 
and responsibility are shared by all Individuals Involved In the program. 

Since the quality monitoring process was first Implemented, program schools selected to 
participate In the process have met with much success In using evaluation results for program 
Improvement. Achievement gains have invariably resulted from the strong commitment of the 
program personnel to this objective, in addition, several other positive outcomes have occurred 
when participating schools are In their second or third year of quality monitoring: 

1 . The district's role in examining evaluation results has diminished as school level staff begin 
to analyze evaluation results themselves. 

2. Program staff have begun to participate In program Improvement plans with a positive 
attitude focusing on solutions rath 3r than problems. 

3. Most Importantly, a self-monltoring attitude has begun to prevail. School level staff are 
now better able to Identify problem areas. This Is evident In quality monitoring sessions 
where school level staff determine the priorities for the year and problem areas to be 
discussed. 

Thus, the quulity monitoring process has encouraged problem solving in an objective and 
systematic manner. With this new perspective, positive changes have continued to occur in the 
program schools. 

As new concerns arise during the school year, new action plans are developed. Some of the 
changes occur immediately; others are made later in the school year. Outcomes reported by 
school level staff are reviewed by district staff at the beginning of the following school year as a 
new quality monitoring cycle begins. 

The quality monitoring process requires the involvement of all program staff. On an on-going 
Lasis. program staff share information, insights and evaluations with the district resource teachers 
and other colleagues. The involvement of the district resource teachers as team members 
provides the opportunity to determine what is really occurring in the classrooms. 

The interactions among the longitudinal database, the annual program evaluations and the quality 
monitoring process have created a set a powerful program processes and highly positive student 
outcomes during the 1985-88 school years. We discuss these in the following sections, using the 
1987-88 school year as an example. 
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Program Characteristics 



Enrollment 

For the 1987-88 school year, the program served 3,750 students in grades K through 12. As In the 
two preceding years, there was a fairly even distribution of participants across grades, although 
heavier concentrations were found In kindergarten through grade 10. Also, there were slightly 
more students in grades K-6 (52 %) than in grade 7<12 (48 %). 

Ethnicity 

Among program partlcipan'is, the single largest ethnic group was Filipino. These youngsters 
represented 31 .0 percent of the elementary and 35.1 percent of the secondary SLEP students. At 
the elementary level, the other major ethnic groups were Indo-Chinese (19.2%), Chinese (14.3 %), 
Samoan (13.7%) and Korean (10.0%). At the secondary level, the other major ethnic groups were 
Chinese (24.2%), Indo-Chinese (12.5%), Korean (10.3%) and Samoan (10.0%). 

Language Groups 

llokano speaking students represented 20.2 p»jrcent of th iotal SLEP enrollment In 1987'88. Other 
major language groups, in order of size, were Cantonese (13.7%), Samoan (1 1.2%), Korean (9.4%) 
and Vietnamese (8.7%). There were 18 other language groups. 

Program Participants 

A majority (61.3%) of the SLEP students were in low language dominance categories. There were 
slightly more male students at both the elementary (54.9%) and secondary (53.5%) levels. A 
significant portion (15.7%) of the students were also in the Chapter 1 program. More than one-half 
of the elementary (63.9%) and secondary (50.8%) students received no other special services. 
About one-third (32.3%) of the elementary and one-half (55.2%) of the secondary students had 
been In the program for two or more years. Approximately 38 percent of the elementary students 
and 17 percent of the secondary students were first year participants. 

Program Setting 

About one-half (51 .9%) of the SLEP schools were located In urban Honolulu. The others were In 
suburban areas. A significant number (49.0%) of these schools used the pull-out setting. The 
remainder mostly used a combination (e.g., pull-out and In-class) of Instructional settings. In 1987- 
80, there was a decrease in the use of the pull-out setting compared with the preceding years. 
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Program Staff 

In 1987-88, a high percentage (25.8%) of the Instructional staff consisted of part-time ESL teachers. 
Other categories of staff Included: 

0 Full-time ESL teachers (1 7.4%) 

0 Part-time bilingual teachers (20.6%) 

0 Full-t!me bilingual teachers (10.3%) 

o Half-time ESL teachers (6.5%) 

0 Half-time bilingual teachers (9.0%) 

0 ESL/bilingual educational assistants (6.4%) 

Most of the program staff (74.4%) had b :en with the program for two or more years. The longevity 
of service was even more pronounced among teachers at the secondary grades where 66.1 
percent of the staff had been with the program for three or more years. A predominant majority 
(87.3%) of the program staff had baccalaureate or higher degrees. A significant number of these 
staff (58.1% at the elementary grades and 23.5% at the secondary grades) majored in elementary 
education. A relatively low percentage (9.3% and 13.7 % for elementary and secondary grades, 
respectively) listed ESL/TE30L as their academic area In college. Over one-hpJf (65.7%) of the 
Instructional staff were fluent in one or more languages other than English. Multi-lingualism was 
particularly evident at the secondary level wherp most (77.2%) of the staff spoke one or more 
languages besides English. 

Type of Instruction 

ESL was the most common type of service provided to SLEP students. This was particularly true 
at the elementary grades where 31 .5 percent of the classes offered ESL instruction. At the 
secondary level, ESL instruction was offered In 17.9 percent of the classes. At the elementary 
level, 10.9 percent of the classes received bilingual instruction. At the secondary level, bilingual 
instruction was offered In 7.1 percent of the classes. In the rest of the SLEP classes, a combination 
of these services was offered to the students. 

Use of English 

In a majority (61 .7%) of the SLEP classes, English was used almost all the time. In 1 7.4 percent of 
the ciassos, English was used about three-fourths of the time. Another 7.4 percent used English 
about half the time. The small remainder reported using English less than half the time. 
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Teacher-Student Ratio 

Although one-on-one Instruction was used In some cases (2.0%), the typical Instructional group 
sizu was between 5 to 10 students. Approximately 22 percent of the Masses reported teacher- 
student ratios ranging from 1 :2 to 1 :5. Another 46 percent had ratios falling between 1 :6 and 1:10. 
Ratios ranging from 1 :1 1 to 1 :15 were found In 18 percent of the classes. The small remainder 
reported ratios higher than 1 :16. As a general trend, the elementary grades enjoyed more 
favorable ratios (smaller groups) than the secondary grades. 

Length of Instruction 

Program students received SLEP Instruction five days a week At the elementary grades, an 
average of 75 minutes of Instruction was provided per day. The secondary students received 
slightly less Instruction, averaging about 68 minutes each day. 

TIme-on-Task 

For over 80 percent of the class time the SLEP students were reported to be engaged In active 
learning (e.g., written assignment, silent reading, listening to teacher lecture, reading aloud and 
class discussion). Close to 36 percent of the time was spent on activities requiring Interaction 
between the teacher and students. Time spent on non-engaged activities (e.g., getting ready for 
learning tasks, socializing, watching others, being disciplined, playing a game or being out of the 
room) was relatively brief. The elementary and secondary classes showed a highly similar pattern 
of time use. 

Class Climate 

The class climate Inventory measures such factors as ccheslveness, goal-direction, clarity, 
satisfaction, openness, reward, cooperation, communication, warmth, formality, favoritism, apathy 
and cllqueness. The data suggest a generally positive class climate In the SLEP program. There Is 
also some evWence that the elementary grades enjoyed a more positive climate than the 
secondary grades. For example, a higher degree of uniformity In Instruction was evident at the 
secondary level than the elementary level, suggesting a lower degree of Individualized Instruction 
at the secondary level. ' 
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Program Implementation 



Data obtained from the program implementation checl<list suggest that in most instances a high 
fidelity implementation of the SLEP program has occurred In the participating schools. The 
checl<list was administered by district level staff who had an intimate l<nowledge of the program. 
These staff used multiple sources of information to assess the status of program implementation in 
a sample of 9^ SLEP classrooms. These sources included document reviews, interviews with 
school level staff, and onsite observations. 6&sed on a convergence of evidence, each critical 
aspect of program operations received one of hree status ratings: not implemented (1). partially 
implemented (2). or fully implemented (3). The 24 critical program elements covered six domains 
of program operations. For the 1987-88 school year, the average ratings were as follows: 

Domain Average Rating 

Instructional approach 2.74 

Diagnosis 2.83 

Student grouping 2.79 

Monitoring 2.87 

Staff development 2.77 

Parent involvement 2.24 

Among the consistently favorable patterns of high ratings, monitoring was the strongest domain of 
program operations. A relatively wdal< area was parent Involvement. A more Indepth lool< at the 
specific elements of Instruction revealed that the use of the student's own language In an 
educationally supportive manner was a relatively weal< are? with a rating of 2.61 . In parent 
involvement, the lower ratings pointed to two relatively wei. areas: (a) parents assisting students 
in class related acth/ities by serving as resource persons, volunteer tutors, or translators; (b) 
parents being provided with materials and/or techniques to develop skills in helping their children. 
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Program 'mpact 



Norm-referenced Test performance 



Achievement data showed a very favorable student performance pattern during the 1985-88 school 
years. Positive performance occurred In oral language proficiency as njeasured by the BINL, In 
reading, language arts and mathematics as measured by the MAT, and In school grades for 
reading, language arts, mathematics and speaking/listening. 

Specifically, program students made substantial NCE gains on the BINL over the three-year period. 
Moreover, the achievement gains Increased from year to year. A similarly favorable performance 
pattern was also evident with respect to reading, language arts and mathematics as measured by 
the MAT. Table 1 presents a summary of the achievement data. 
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Table 1 



NCE Scores of Program Students on the BINL and the MAT 

Area 1985-86 1986-87 1987-88 
BINL 

Pretest 74 71 65 

Posttest 82 84 87 

Gain 8 13 22 

MAT Reading 

Pretest 26 26 22 

Posttest 29 31 29 

Gain 3 5 7 

MAT Language Arts 

Pretest 28 27 23 

Posttest 32 33 29 

Gain 4 6 6 . 

MAT Mathematics 

Pretest 37 38 33 

Posttest 40 42 38 

Gain 3 4 5 



Note. For each school year, data were based on approximately 1,700 program 
students for whom there were matched pretest and posttest scores. 




18 



20 



It should be noted that the BINL was normed on regional (mainly California) samples of students 
having similar backgrounds as SLEP students. In addition, there Is a close vn\ Ich between the test 
contents and the SLEP curriculum. The relatively high NCE scores and NCE gains are 
undoubtedly a result of these factors. The MAT, on the other hand, was normed on a nationally 
representative sample of the U.S. student population. Also, a new (1985) version of the MAT was 
used In 1987-88. The lower NCE scores for that school year were primarily due to the use of the 
new MAT norms. 

A closer lool< at the proportion of program students nruiking positive NCE gains revealed that a 
predominant majority of the students made gains on the BINL. The MAT data showed that a large 
majority of the participants nfiade positive NCE gains. Again, Increasingly higher percentages of 
program students made positive gains on both the BINL and the MAT over the three-year period. 
Table 2 presents a summary of the data. 



Table 2 



Percent of Program Students Making Positive NCE Gains 

MAT 

School Year BINL Reading Lang. Arts Math 



1985-86 


81.3 


57.8 


59.1 


53.3 


1986-87 


81.0 


59.8 


61.9 


59.4 


1987-88 


86.7 


69.9 


66.6 


64.3 



Note. For each school year, data were based on approximately 1 ,700 students for whom there 
were matched pretest and posttest scores. 
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Criterion-Referenced Performance 

In addition to norm-referenced test data, the program uses a Student Record Profile (SRP) system 
to gather data on specific language objectives completed by each program student. These 
objectives cover four domains of language skills: syntax, semantic, morphemic and phonic. There 
are 22 specific objectives In syntax, 20 In semantic, 7 In morphemic and 4 In phonic. For each o* 
the three school years, the data showed that the average program participant completed a 
majority of the objectives. The completion rates were higher In morphemic and phonic where the 
average student niastered about two-thirds of the objectives during each school year. Also, as one 
would expect, the secondary students demonstrated substantially higher rates of completion than 
their elementary counterparts. 

The SRP system also Includes data on social adjustment; culture, basic survival, school survival 
and community survival. There are 5 specific objectives In culture survival, 7 In basic survival, 16 In 
school survival and 1 1 in community survival. During each school year, the average SLEP student 
completed a majority of the objectives. As one would expect, secondary students dki 
considerably better than their elementary counterparts In mastering these objectives. 

School Grades 

The longitudinal database includes data on grade point averages of program students In four 
subject areas: mathematics, language arts, reading and speaking/listening. Since grading 
practices differ from school to school, all grade designations are converted to a five-point scale: 

1 = Failing 

2 - Barely passing 

3 = Satisfactory 

4 = Very good 

5 = Excellent 

The data suggest that as a group, the SLEP students were performing satisfactory school work In 
the four subject areas. In all Instances, the failure rates hovered around 5 percent or less. About 
20 percent of the students showed "very good" performance. Approximately 6 to 8 percent were 
doing "excellent" work In language arts, reading and speaking/listening. Close to 13 percent had 
"excellent" grades In mathematics. 
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Performance of Exited Students 



Approximately 150 students exited the program each school year. There Is strong evidence that 
•f these students were performing well In the regular classroom as well as on norm-referenced 
tests. Specifically, In all Instances, a predominant majority of the students were doing satisfactory 
or better work In reading, language arts, mathematics and speaking/listening. Generally, less than 
3 percent of the exited students failed In any of these subject areas. Close to one-third of the 
exited students received Very good" grades In language arts, reading and speaking/listening. 
More than a quarter of the exited students were doing "excellent" work In mathematics. 

Moreover, the exited students were able to continue to make positive NCE gains in reading, 
language arts and mathematics as measured by the MAT. Although these gains were relatively 
small, they were consistent over the three-year period. Table 3 presents a summary of the data. 
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Table 3 



NCE Scores of Exited Students 



Area 1985-86 1986-87 1987-88 

MAT Reading 

Pretest 44 46 37 

Posttest 46 49 44 

Gain 2 3 7 

MAT Lang. Arts 

Pretest 47 49 41 

Posttest 49 52 49 

Gain 2 3 8 

MAT Mathematics 

Pretest 54 57 54 

Posttest 56 59 57 

Gain 2 2 3 



Note. For each school year, data were based on approximately 150 exited students for whom there 
were matched pretest and posttest scores. The data reflect achievement status a year after the 
students exited the program. 

The 1985 version of the MAT was used in 1987-88. The lower NCE scores were primarily due to 
the use of the new MAT norms. It is also noteworthy that the exited students scored very close to 
the national norms (50 NCE) In reading and language arts. Their scores were slightly above the 
national average In mathematics. 
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Perhaps more significaritly, a high percentage of the exited students continued to mal<e positive 
NCE gains on all three MAT subtests. As shown in Tabie 4, from one-half to two-thirds of the exited 
students showed positive NCE gains one year after they exited the program. 



Table 4 



Percent of Exited Students Making Positive NCE Gains 



Area 1985-86 1986-87 1987-88 



MAT Reading 


55.1 


62.3 


69.3 


MAT Lang. Arts 


56.0 


61.8 


67.3 


MAT Mathematics 


53.5 


49.3 


58.2 



Note. For each school year, data were based on approximately 150 exited students for whom there 
were matched pretest and posttest scores one year after they exited the program. 
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Parent Activities 



ERIC 



Program records showed that a wide range of activities was provided to SLEP parents each school 
year. Examples included: 

0 Open house/orlentatlon meeting 

0 Cultural activities (e.g., Mothers' Day, Chrlstrnt. I'tural presentation, choral 
performance, cooi<ing demonstratlori) 

o Parent lnservice/wori<shop/class (e.g., computers, self-esteem, home reading, 
booi<mai<lng, conversational English) 

0 Parent/teacher/school conference 

0 Bool< fair 

0 Excursion (e.g., library tour, field trip) 

o Student achievement awards program 
In addition, adult education classes were offered at four school sites to help parents to: 

o learn to speal<, read and write English; 

o prepare for citizenship and driving tests; 

0 enroll In higher education or employment training; 

o Improve self-image and set goals for better living, and 

0 learn another langucsge. 
Parent learning centers were established at nine program sites to: 

0 assist parents In understanding Hawaii's educational system; 

0 provide educational activities for parents so that they are able to help their children at 
home with school work; 

0 help parents learn survival skills which enable them to function better In the community; 

0 help parents learn basic communication skills In English so they can understand and 
participate in school activities; 

0 assist parents In understanding other cultures In Hawaii through multicultural activities; 
and 

0 help parents learn how to use computers so that they can assist their children. 

24 

26 



Summary and Coricluslons 



Recently, bilingual education has received much attention from researchers as well as the federal 
government (e.g.. Baker & de Kanter. 1983; Berke. 1983; Willlg, 1985). The attention has focused 
on both program Implementation and outcomes of educational servbas provided to Increasingly 
large numbers of youngsters with limited English proficiency. One such effort Is Implemented in 
the Honolulu district which has a high concentration of low-income and immigrant families. The 
SLEP program seeks to provide a smooth transition for recent immigrant students to adjust to the 
dominant culture and regular school life In the United States. Its primary objective is to improve 
the students' English language skills so that they may function effectively in the regular classroom. 
The program offers special educational services In 54 schools, serving some 4,000 students in 
kindergarten through high school grades. 

Drawing on the effective schools research base, the district has Implemented a well-integrated set 
of essential program components. It has demonstrated a successful effort in helping children of 
limited English profic'ency to adjust to the malnstraain American culture. 

Evidence of positive program impact with respect to norm-referenced achievement is strong and 
consistent. A predominant majority of the program students made positive NCE gains in oral 
language as measured by the BINL. A large majority also made positive NCE gains in reading, 
mathematics, and language arts as measured by the MAT. narrowing the achievement gap 
between SLEP students and their non-SLEP counterparts. Other data show that the program 
students were performing well in the regular school program. Most received satisfactory or better 
grades in mathematics, reading, language arts and speaking/listening. On the average, less than 
5 percent of the program students received failing grades In these subject areas. Perhaps more 
remarkably, in most instances, a large majority of exited students continued to make achievement 
gains on norm-referenced tests and were performing well in the regular program. Over a three- 
year period, both current and exited students showed an Increasingly positive performance 
pattern. 

The essential program elements responsible for the positive Impact are clearly documented by 
district staff and seem easily transportable to other locations. It is likely that the implementation of 
these program activities in other school systems will bring about similarly favorable results. 
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